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1.  Introduction 


1.1  Problem  Statement  (Driving  -  Drowsiness  Causes  Crashes) 

The  significant  role  of  drowsiness  in  motor  vehicle  accidents  is  a  serious  concern  for  both  the 
public  sector  and  the  military.  In  1990,  the  U.S.  National  Transportation  Safety  Board  (NTSB) 
determined  fatigue  to  be  the  most  frequent  factor  in  U.S.  fatal  truck  crashes  (1).  Another  NTSB 
study  in  1995  found  that  58%  of  the  107  non-fatal  single-vehicle  roadway  departure  truck 
crashes  were  related  to  fatigue  (2).  Dinges  et  al.,  cites  that  fatigue  can  be  at  least  partially 
attributed  to  2%-23%  of  all  crashes  (3-7),  4%-25%  of  single-vehicle  crashes  (8,  9),  10%-40% 
of  long  motor  way  crashes  (10,  11),  and  15%  of  fatal  single- vehicle  truck  crashes  (S). 

Studies  on  the  incidence  of  drowsiness-related  motor  vehicle  accidents  indicate  that  drivers  are 
generally  ineffective  in  judging  performance  degradation  due  to  drowsiness  until  they  are 
severely  impaired  to  the  point  of  head  bobbing  (12).  This  indicates  that  real-time  drowsiness 
detection  systems  are  needed  for  the  typical  driver  to  enhance  their  ability  to  judge  drowsiness 
onset. 

1.2  Objective  of  this  Project 

Defining  the  drowsiness  state  is  subjective  and  beyond  the  scope  of  this  report.  Rather  than 
attempting  to  assess  drowsiness,  this  report  reflects  a  portion  of  a  larger  project  proposing  to  fuse 
multiple  sensor  modalities  in  order  to  predict  fatigue-induced  driver  performance  through  the 
integration  of  electroencephalography  (EEG),  eye-tracking  sensors,  and  behavioral  monitors.  As 
an  initial  step  towards  the  prediction  of  driver  performance  degradation  as  a  result  of  drowsiness, 
the  work  described  in  this  report  aims  to  complete  the  following  two  objectives: 

1.  Implement  an  improved,  fieldable  algorithm  for  the  automatic  detection  of  alpha  bursts. 
(Accomplished) 

2.  Quantify  cross-correlations  between  eye-tracking  features,  alpha  bursts,  and  fatigue-related 
driver  performance  measures.  (Partially  Accomplished) 

This  report  describes  the  methodologies  we  employed  to  accomplish  the  automatic  detection  of 
alpha  bursts  in  EEG,  the  eye-tracking  analysis,  the  behavioral  analysis,  and  the  cross  sensor 
correlation  analysis.  This  report  also  highlights  the  effectiveness  of  using  these  methods, 
discussing  the  reasons  that  some  of  the  methods  used  were  ineffective  within  this  particular 
implementation. 

1.3  State  of  the  Art  Fatigue  Systems 

Several  real-time  drowsiness  monitoring  systems  have  been,  or  are  in  the  process  of  being, 
developed,  but  the  technology  has  not  yet  reached  an  acceptable  level  of  accuracy  for  the  Army. 
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This  is  due  in  a  large  part  to  the  difficulty  in  defining  the  abstract  concept  of  “drowsiness”  in 
clinically  measurable  terms.  Some  promising  technologies  for  predicting  behavioral  performance 
indicators  of  drowsiness  are  EEG,  eye-tracking  measures,  and  driver  performance.  Each  of  these 
sensor  modalities  has  been  individually  correlated  to  the  state  of  drowsiness.  Spectral  power  has 
a  characteristic  distribution  in  EEG  recordings  of  the  parietal  and  occipital  regions  of  the  brain 
during  the  drowsiness  state  as  compared  to  REM  sleep  and  other  states,  as  seen  in  figure  1  (13). 
Furthermore,  using  EEG,  Picot  et  al.  (14)  were  able  to  achieve  85%  sensitivity  and  20% 
specificity  in  assessing  drowsiness.  PERCLOS  of  the  eye,  defined  as  the  percentage  of  time  the 
eye  is  at  least  80%  closed,  has  been  highly  correlated  to  drowsiness;  it  yields  a  20-min  bout-to- 
bout  coherence  Pearson  correlation  coefficient  of  0.878  (15).  The  eye-tracking  measures  of  pupil 
constriction  and  oscillation  are  also  valid  measurements  for  drowsiness  detection  (16).  Many 
vehicle-based  drowsiness  detection  systems  use  behavioral  measures,  such  as  standard  deviation 
of  lateral  lane  position  (STDLAT),  to  assess  sleepiness.  STDLAT  has  been  correlated  to  the 
Karolinska  Sleepiness  Scale  (KSS),  with  a  chi-squared  value  of  11  (p  <  0.001)  (17).  For  this 
study,  we  look  at  the  cross  correlation  of  behavioral,  eye-tracking,  and  EEG  as  a  means  to 
understanding  how  multiple  sensor  modalities  can  be  used  to  improve  the  sensitivity  and 
specificity  of  drowsiness  prediction. 


Figure  1.  Spectral  power  contribution  of  sleep-alpha  variants 

(drowsiness  alpha  activity  and  REM-alpha  bursts)  over 
frontal,  central,  parietal  and  occipital  regions.  Note  the 
higher  spectral  contribution  of  the  slowest  components 
(7. 8-8. 6  Hz)  during  REM  alpha  bursts  as  compared  with 
drowsiness-alpha  activity  (13). 
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1.4  EEG — Definition  and  Relation  to  Drowsiness 

EEG  is  the  study  of  the  electrical  signals  generated  by  the  human  brain.  Most  notable  to  the  field 
of  electroencephalography,  an  electroencephalogram  is  a  device  that  simultaneously  records  the 
voltage  across  a  reference  electrode  (or  set  of  reference  electrodes)  and  many  additional 
individual  electrodes,  all  positioned  on  a  subject’s  scalp.  Since  its  initial  use  in  1929, 
physiologists  have  inferred  that  an  EEG’s  electrical  readings  are  a  measure  of  the  electrical 
activity  of  the  brain  (18).  The  changes  in  electrical  potential  of  a  particular  electrode  over  time 
have  been  interpreted  as  the  temporal  changes  in  neurological  response,  which  spatially 
correspond  to  a  specific  region  of  the  brain.  The  distribution  of  electrodes  over  the  scalp  provides 
a  means  for  comparison  of  neurological  responses  in  different  regions  of  the  brain  at  a  particular 
instance  in  time.  Most  current,  lab-based  systems  EEG  devices  consist  of  16-256  electrodes 
snapped  into  a  form- fitting  cap  placed  on  the  subject’s  head,  and  require  conducting  gel  to 
facilitate  an  electrical  connection  between  the  scalp  and  the  electrodes.  Systems  designed  for 
gaming  tend  to  have  even  fewer  electrodes  and  many  utilize  dry  electrode  technology. 

1.5  Current  EEG  Analysis  Approaches  to  Measure  Drowsiness 

Recent  research  has  explored  the  correlation  of  EEG  signals  with  behavioral  indicators  of 
drowsiness.  Studies  have  found  that  the  onset  of  drowsiness  is  typically  associated  with  an 
increase  in  brain  activity  in  the  alpha  and  theta  frequencies  (8-13  and  4-7  Hz,  respectively) 
coinciding  with  a  decrease  in  brain  activity  in  the  beta  band  (14-60  Hz)  (18,  19). 

EEG  recordings  from  subjects  participating  in  a  driving  simulation  were  obtained  to  assess  the 
viability  of  using  a  means  comparison  test  to  monitor  the  (a  +  0)/p  ratio  as  a  drowsiness 
detection  algorithm  (14).  This  method  did  not  predict  driver  performance;  instead,  this  algorithm 
attempted  to  assess  whether  or  not  a  subject  was  drowsy.  The  algorithm’s  output  was  compared 
to  an  expert’s  drowsiness  assessment  of  many  20-s  EEG  segments,  boasting  85%  sensitivity  and 
20%  specificity  for  assessing  drowsiness.  The  main  advantage  of  this  algorithm  is  that  it  can  be 
run  in  near  real  time  on  a  single  EEG  channel,  making  it  a  fieldable  algorithm.  However,  this 
algorithm  is  limited  in  that  it  attempts  to  assess  the  complex  physiological  state  of  drowsiness. 
Furthermore,  power  ratios  are  not  ideal  indicators  for  detecting  fatigue  due  to  the  detrimental 
effects  that  artifacts  and  noise — which  are  present  in  field  environments — have  on  these  metrics 
(20). 

Simon  et  al.  (20)  proposed  that  the  detection  of  alpha  bursts  is  a  more  robust  measure  of  fatigue 
in  the  presence  of  artifacts.  Alpha  bursts  (pictured  in  figure  2)  are  characterized  by  narrowband 
increases  in  power  within  the  alpha  frequency  band  (8-13  Hz)  and  are  typically  observed  in  a 
subject’s  EEG  signal  when  transitioning  between  drowsiness  and  the  early  stages  of  sleep  (14, 

19,  21).  Simon  et  al.  compared  alpha  burst  frequency  and  alpha  power  in  subjects  during  a  real 
driving  task.  The  frequency  of  alpha  bursts  increased  by  90%  from  the  awake  to  drowsiness 
states  whereas  alpha  power  only  increased  by  32%.  The  findings  indicate  that  alpha  burst 
detection  provides  a  more  accurate  drowsiness  assessment  than  power  measures  under  real 
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driving  conditions.  The  main  advantage  of  this  algorithm  is  that  it  uses  a  single  EEG  channel  and 
is  relatively  robust  in  the  presence  of  noise  and  artifacts.  However,  this  algorithm  does  not  work 
in  real  time  and  is,  therefore,  not  field-implementable.  In  this  study,  we  avoided  the  issue  of 
defining  the  drowsiness  state.  Instead,  we  accepted  that  alpha  burst  frequency  is  a  robust 
indicator  of  the  transition  between  being  awake  and  asleep  and  intended  to  improve  upon  the 
Simon  et  al.  alpha  burst  detection  algorithm  by  implementing  it  in  real  time  and  optimizing  its 
parameters  (20). 

P3 
Pz 
P4 
Ol 


Figure  2.  Example  of  alpha  burst  in  the  parietal  and  occipital  regions  of  the  brain  (20). 

1.6  Eye-tracking 

Recent  advancements  in  eye-tracking  technology  have  enabled  the  eye  to  be  monitored  remotely, 
making  this  device  the  least  intrusive.  Eye-tracking  techniques  usually  consist  of  a  table  or 
dashboard  mounted  camera  with  software  that  uses  feature  extraction  for  each  image  to  record 
measures  such  as  PERCLOS,  pupil  diameter,  and  gaze  distribution  (GAZEDIS).  As  shown  in 
figure  3,  there  has  been  a  substantial  amount  of  causal  interaction  research  that  has  linked  a 
number  of  different  exposures  to  potential  fatigue  outcomes  (22).  Ji  et  al.  (22)  have  used  this  to 
develop  a  statistical  Bayesian  Network  causal  relationship  between  exposure  and  outcome. 


Figure  3.  Work  by  Ji  et  al.  (22)  highlighting  the  causal  interactions  and  outcomes  regarding  fatigue. 
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1.7  Current  Eye-tracking  Systems  to  Measure  Drowsiness 

Drowsiness  is  an  inherently  difficult  physiological  construct  to  experimentally  define.  Many 
researchers,  instead,  correlate  behaviors  or  physiologically  measurable  factors  to  outcomes,  thus 
obviating  the  need  to  strictly  define  “drowsiness.”  The  Federal  Highway  Administration  has 
determined  the  most  effective  measure  of  alertness,  the  inverse  of  drowsiness,  to  be 
PERCLOS — defined  as  the  percentage  of  the  time  that  the  eye  is  at  least  80%  closed  (1).  A 
Department  of  Transportation  study  strongly  correlated  PERCLOS  with  psychomotor  vigilance 
test  (PVT)  performance  lapses,  which  is  a  well-documented  good  test  for  driving  alertness 
performance  measure  (15).  More  specifically,  PERCLOS  is  defined  as  the  percentage  of  time 
within  a  minute  that  the  eye  is  more  than  80%  closed  (23)  and  is  considered  the  gold  standard  for 
measuring  alertness  (24).  The  disadvantage  to  using  PERCLOS  as  a  real-time  drowsiness 
measure  is  that  calculating  PERCLOS  requires  a  time-delay  on  the  order  of  minutes.  Any  time- 
delay  decreases  the  efficacy  of  a  fielded  drowsiness  monitoring  system,  as  driver  impairment 
feedback  notification  will  be  delayed  as  well.  An  incident  caused  by  drowsiness-related 
impairment  could  occur  in  the  minutes  required  to  process  the  PERCLOS  data;  however,  except 
in  extreme  cases,  most  people  do  not  experience  drastic  changes  in  alertness  over  a  few  seconds. 
Instead,  most  people  experience  a  gradual  change  over  many  minutes  from  alert  to  not  alert.  As 
such,  PERCLOS  measures  that  have  a  measurement  time  of  several  minutes  should  be  applicable 
for  most  driving-based  applications. 

Many  additional  eye-tracking  measures  have  been  investigated  as  indicators  of  drowsiness, 
including  pupil  diameter,  GAZEDIS,  and  saccadic  velocity.  The  study  by  Merritt  (16)  found  that 
measures  of  pupil  constriction  and  oscillation  were  valid  metrics  for  this  application.  Russo  et  al. 
(25)  found  a  correlation  between  simulation  car  crash  incidence  with  decreases  in  saccadic 
velocity  (R‘  =  77%)  and  increases  in  pupil  constriction  latency  (R~  =  88%).  Jain  et  al.  (26)  used 
the  sum  of  spectral  power  from  0  to  0.8  Hz  of  pupil  diameter  for  82-s  windows  as  a  measure  of 
“pupillary  unrest”  (effect  size  =  0.9).  In  this  study,  we  investigated  cross  correlating  the  eye¬ 
tracking  features  of  pupil  diameter  and  gaze  distribution,  since  these  measures  can  be  calculated 
almost  instantaneously.  Since  drowsiness  comes  in  cycles  with  both  low  and  high  frequency 
components  (2,  27),  we  investigated  both  short  term  (minute-to-minute)  and  long  term  (10  min 
bout-to-bout)  time  intervals.  A  high  degree  of  both  inter-  and  intrapersonal  variation  in  these 
correlates  has  been  previously  established  (1,  28),  so  we  conducted  a  variation  study  both  within 
subject  datasets  and  between  subject  datasets. 

1.8  Behavioral  Indicators  of  Fatigue 

A  number  of  driving  performance  measures  have  been  established  as  drowsiness-related.  A  very 
commonly  used  metric  is  lane  drifting,  measured  by  the  SDLAT  (2,  29).  A  linear  trend  of  the 
SDLAT  as  a  function  of  the  KSS  yielded  the  chi-squared  value  of  38  (p  <  0.001)  with  an  average 
increase  of  0.032  m  for  each  level  of  the  KSS  (28).  Adding  a  squared  component  further 
improved  the  model  fit  (chi-squared  value  is  1 1,  p  <  0.001),  suggesting  a  curvilinear  relationship 
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between  the  KSS  and  SDLAT  (25).  Reaction  time  is  also  a  very  important  driving  performance 
factor  (30)  that  is  highly  correlated  to  drowsiness  (31).  In  this  study,  we  investigated  the 
drowsiness  measures  SDLAT  and  standard  deviation  of  acceleration. 

In  addition,  we  investigated  the  drowsiness  measures  SDLAT  and  standard  deviation  of 
acceleration  because  these  measurements  were  continuous  rather  than  event- specific  metrics,  and 
they  provided  enough  data  for  comparison  with  other  sensor  modalities.  Reaction  time  was 
rejected  as  a  viable  measure  of  drowsiness  in  this  case,  because  measurements  of  reaction  time 
occurred  at  discrete  events  that  often  did  not  overlap  with  alpha  bursting  or  “good”  windows  of 
eye-tracking  data;  therefore,  statistically  significant  analysis  using  reaction  time  could  not  be 
performed  on  the  limited  dataset  (32).  Instead,  we  explored  the  possibility  of  using  the  standard 
deviation  of  acceleration  as  a  substitute  for  reaction  time  analysis.  Although  this  metric  has  not 
been  correlated  with  drowsiness,  we  assumed  that  standard  deviation  of  acceleration  can  be 
interpreted  as  a  continuous  measure  of  a  driver’s  instantaneous  reaction  to  random  deviations 
from  the  target  speed. 


2.  Methodology 


In  order  to  accomplish  our  objectives,  two  distinct  phases  of  the  project  were  realized,  depicted 
in  figure  4.  The  initial  data  from  each  of  the  three  sensor  modalities  were  pre-processed,  parsed, 
and  ground-truthed.  Once  this  pre-processing  was  finished,  an  alpha  burst  detection  algorithm 
with  the  ability  to  be  implemented  in  near  real  time  was  performed  post-hoc  on  the  data  and 
improved  through  optimization.  Concurrently,  correlations  between  the  different  sensor 
modalities  were  established  and  quantified. 
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Figure  4.  This  overarching  project  map  depicts  the  relationships  between  three  branches  of  sensor  data:  EEG,  eye¬ 
tracking,  and  driving  performance. 

2.1  Driving  Simulation 

The  data  used  in  this  report  were  taken  from  a  Driving  Simulation  Pilot  Experiment  performed 
by  the  Human  Research  and  Engineering  Directorate  (HRED)  of  the  U.S.  Army  Research 
Laboratory  in  summer  2012  at  Aberdeen  Proving  Grounds,  MD.  Twenty-three  male  subjects 
participated  in  the  experiment.  The  simulation  consisted  of  a  driving  task  for  which  participants 
were  asked  to  drive  straight  for  a  period  of  45  min  to  1  hour  after  a  15-min  calibration  drive. 

The  simulator  consisted  of  a  video  game  mock-up  of  a  racecar  wheel  that  controlled  the  virtual 
movements  of  the  subject’s  car  on  a  computer  monitor.  A  representation  of  the  simulation  field 
of  view  is  shown  in  figure  5.  The  simulator  also  recorded  various  driver  activity  metrics, 
including  lane  position,  speed,  and  acceleration.  Periodically  throughout  the  experiment,  the 
simulator  delivered  a  “perturbation  event,”  which  caused  a  virtual  disturbance  in  the  heading  of 
the  subject’s  vehicle.  The  amount  of  time  the  subject  took  to  initialize  a  response  is  referred  to 
as  the  perturbation  event  “reaction  time”  in  the  analysis. 


7 


Figure  5.  Representation  of  the  simulation  field  of  view. 

2.2  Automatic  Detection  of  Alpha  Bursts 
2.2.1  BioSemi  System 

Throughout  the  driving  simulation,  the  subject’s  brain  activity  was  monitored  via  a  BioSemi 
EEG  system  (figure  6).  The  EEG  hardware  used  in  this  study  consists  of  64  active  channels  and 
8  additional  channels,  which  include  electrooculography  (EOG)  channels  and  reference 
electrodes.  Each  channel  recorded  data,  which  were  analyzed  post- simulation  using  Matlab  and 
the  EEGLab  toolbox  (33). 


Figure  6.  The  64-channel  BioSemi  EEG  System  (33). 

2.2.2  Establishing  Ground  Truth 

In  order  to  implement  a  fieldable  algorithm  to  detect  alpha  bursts,  a  means  of  assessing  the 
algorithm  needed  to  be  established.  There  was  no  standard  protocol  for  identifying  alpha  bursts. 
Rather,  an  EEG  expert  is  needed  to  manually  identify  “events”  from  EEG  features.  Part  of  the 
scope  of  this  report  includes  the  development  of  a  standardized  protocol  for  establishing  ground 
truth  for  EEG  features.  The  procedure  for  establishing  ground  truth  is  outlined  here: 

1.  Using  a  scale  from  0-3  (0:  No,  1:  Maybe,  2:  Probably,  3:  Textbook  Example),  two 

independent  reviewers  examine  all  of  the  EEG  data  and  record  the  start  and  end  times  of 
regions  corresponding  to  alpha  bursts,  according  to  the  following  criteria: 
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•  Presence  of  medium  to  high  amplitude  activity  in  the  alpha  range  (8-13  Hz)  located  in 
the  occipital  and  parietal  lobes 

•  Duration  of  at  least  0.5  s 

•  Alpha  bursts  separated  by  1  s  or  less  are  combined  into  a  single  burst 

2.  The  two  independent  reviewers  meet  and  examine  all  examples  rating  disagreements 
resulting  from  step  one,  deciding  upon  the  absolute  binary  presence  or  absence  of  an  alpha 
burst  for  each  event.  Any  event  with  a  score  of  1  or  0  is  eliminated  from  the  dataset. 

3.  The  final  list  of  alpha  bursts  for  80%  of  the  subjects  is  sent  to  an  expert  (EEG 
neuroscientist  with  10+  years  of  experience)  for  final  review  and  to  eliminate  false 
positives.  The  expert  then  blindly  reviews  the  remaining  20%  of  subjects  to  establish  a 
false  negative  rate. 

The  implementation  of  Step  3  in  the  procedure  was  reduced  from  80%  of  all  subjects  to  one 
subject  (4.3%)  in  the  initial  implementation  and  will  be  expanded  to  the  full  percentage  in  the 
final  experimental  implementation. 

2.2.3  Original  Algorithm 

To  ascertain  the  correlation  between  alpha  bursts  and  drowsiness,  Simon  et  al.  developed  an 
algorithm  for  the  automatic  detection  of  alpha  bursts.  A  flowchart  representation  of  this 
algorithm  is  pictured  in  figure  7.  To  estimate  the  characteristic  (1/f-*3)  noise  in  the  EEG  signal, 
the  algorithm  applied  an  exponential  fit  to  the  amplitude  spectral  density  (ASD)  of  each  subject’s 
entire  recording  (20,  34).  The  algorithm  epoched  (discretely  segmented)  the  data  for  each  EEG 
channel  into  1-s  windows  with  an  overlap  of  750  ms  across  windows.  To  negate  the  effects  of 
windowing  the  data,  each  window  was  zero-meaned  and  multiplied  with  a  Hamming  window. 
The  short  time  Fourier  transform  (in  its  fast  Fourier  transform  implementation)  was  computed  to 
ascertain  the  spectral  amplitude  density  in  each  window.  The  previously  calculated  noise 
approximation  was  applied  to  the  current  segment  by  multiplying  the  exponential  curve  by  the 
ratio  of  the  integrated  mean  spectrum  of  the  current  window  to  the  integrated  mean  spectrum  of 
the  entire  recording.  The  full  width  at  half  maximum  (FWHM)  of  the  alpha  frequency  peak  was 
then  calculated  according  to  the  following  procedure.  First,  the  two  local  minima  surrounding  the 
peak  were  located.  From  there,  the  “bottom”  of  the  peak  was  calculated  by  averaging  the 
amplitude  of  the  local  minima.  The  FWHM  was  considered  to  be  the  width  of  the  peak  at  the 
height  midway  between  the  “bottom”  and  maximum  of  the  peak.  If  the  characteristics  of  the 
window  met  the  following  criteria,  the  window  was  said  to  contain  an  alpha  burst: 

1.  The  peak  ASD  was  in  the  alpha  frequency  band  (7-13  Hz). 

2.  The  FWHM  of  the  peak  ASD  was  less  than  half  the  bandwidth  of  the  Hamming  window. 
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3.  The  area  in  the  peak  ASD  above  the  noise  estimate  (signal)  was  greater  than  the  area  in  the 
peak  ASD  below  the  noise  estimate  (noise). 


EEG  Channel 


Preprocess  Signal 


Split  Data  into  1  s  windows  with  750  ms 
overlap.  Multiply  by  a  Hamming  Window  and 
Zero  Mean. 


Is  the  Peak  Amplitude  Spectral 
Density  in  the  Alpha  Band? 
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than  Twice  the  Hamming  Window 
Bandwidth? 
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Density  as  Estimate  of  Noise.  Is  Area  in  Peak 
Above  Noise  Greater  than  Noise  Area? 


Yes 


Alpha  Burst! 


Figure  7.  Algorithm  for  automatic  detection  of  alpha  bursts  (20). 

2.2.4  Fieldability  Concerns 

Simon  et  al.,  in  (20)  attempted  to  address  the  fieldability  concerns  of  using  EEG  by  only 
requiring  a  single  channel  as  input  for  this  alpha  burst  detection  algorithm.  It  is  important  to  note, 
however,  that  the  noise  estimation  portion  of  Simon  et  al.’s  algorithm  could  not  be  implemented 
in  real  time  since  it  required  the  entire  EEG  recording.  Although  Simon  et  al.  partially  addressed 
the  fieldability  concerns  of  using  EEG  as  a  drowsiness  detector,  a  crucial  step  of  the  proposed 
algorithm  was  to  estimate  the  noise  across  the  frequency  spectrum  in  the  EEG  signal.  Simon  et 
al.  applied  an  exponential  fit  to  the  mean  spectral  amplitude  density  for  the  entire  recording  in 
each  channel  in  order  to  approximate  the  nonlinear  (1/f  ^)  noise  that  is  inherent  in  EEG 
recordings  (29).  This  curve  acted  as  a  barrier  allowing  one  to  distinguish  between  the  signal  (the 
spectral  amplitude  density  above  the  curve)  and  the  inherent  EEG  noise  (the  spectral  amplitude 
density  below  the  curve).  In  order  to  account  for  inter-subject  and  longitudinal  noise  variability, 
Simon  et  al.  multiplied  the  noise  estimation  by  the  ratio  between  the  integrated  spectrum  of  the 
current  window  with  the  mean  integrated  spectrum  of  the  entire  recording.  Since  this  noise 
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estimation  scheme  required  post-processing  of  the  entire  recording,  the  algorithm  outlined  by 
Simon  et  al.  could  not  be  performed  in  real  time. 

In  order  to  implement  a  near-real-time  alpha  burst  detector,  the  noise  approximation  scheme 
described  by  Simon  et  al.  was  adjusted.  Rather  than  fitting  an  exponential  curve  to  the  entire 
recording  and  then  applying  a  proportional  curve  to  each  window,  as  Simon  et  al.,  did,  an 
exponential  curve  was  instead  fitted  to  each  window  independently.  An  example  window  of  the 
real-time  implementation  of  the  alpha  burst  detection  algorithm  is  shown  in  figure  8. 


Figure  8.  Depiction  of  the  spectral  amplitude  density  of  a  one-second  window 
during  an  alpha  burst.  The  real-time  noise  approximation  overlays 
the  spectral  amplitude  density. 

2.2.5  Algorithm  Optimization 

To  improve  the  real-time  implementation  of  the  alpha  burst  detection  algorithm,  the  EEG  data 
was  split  into  three  subsets:  a  training  dataset,  a  testing  dataset,  and  a  validation  dataset  (about 
70%,  20%,  and  10%  of  the  data,  respectively).  The  EEG  recordings  from  each  subject  were  split 
into  segments  of  roughly  100  s  and  randomly  distributed  into  each  dataset.  The  training  dataset 
was  used  to  choose  the  optimum  channel  and  optimize  the  algorithm.  The  first  step  of  the 
algorithm  optimization  procedure  was  to  run  the  real-time  implementation  of  the  alpha  burst 
detection  algorithm  for  all  64  channels  within  the  training  dataset.  Then  the  sensitivity  (true 
positive  rate)  and  specificity  (true  negative  rate)  were  assessed  for  each  channel  using  ground- 
truth  data.  Treating  the  sensitivity  and  specificity  scalar  values  as  orthogonal  vectors  allowed  us 
to  find  the  optimal  channel.  The  optimal  channel  had  the  minimum  distance  between  the  vector 
<sensitivity,  specificity>  and  the  vector  <1,  1>.  Once  the  optimal  channel  was  discovered,  the 
remaining  optimization  was  specific  to  that  channel. 
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Multiple  parameters  were  independently  optimized  by  running  the  alpha  burst  detection 
algorithm  on  the  best  channel  while  varying  the  values  of  each  parameter — one  parameter  at  a 
time.  The  sensitivity  and  specificity  were  calculated  for  each  value  of  each  parameter  using  the 
training  dataset.  The  same  technique  described  for  finding  the  optimal  channel  was  used  for  each 
parameter;  the  optimal  value  for  the  current  parameter  was  discovered  by  finding  the  value  that 
yielded  the  minimum  distance  between  the  vector  <sensitivity,  specificity>  and  the  vector 
<1,  1>.  This  procedure  was  performed  for  the  following  parameters:  window  length,  overlap 
between  windows,  minimum  alpha  frequency,  and  maximum  alpha  frequency. 

2.2.6  Testing  and  Validation 

Once  the  optimum  values  for  each  parameter  were  found,  three  iterations  of  the  alpha  burst 
detection  algorithm  were  tested:  the  original  algorithm,  the  real-time  implementation  of  the 
algorithm  with  original  parameters,  and  the  real-time  implementation  of  the  algorithm  with 
optimized  parameters.  For  each  iteration  of  the  algorithm,  the  sensitivity  and  specificity  were 
calculated  using  the  testing  dataset. 

A  third  dataset,  the  validation  dataset,  was  used  to  verify  that  the  training  and  testing  datasets 
were  representative  of  all  the  alpha  burst  data.  To  verify  that  the  three  datasets  were  statistically 
equivalent,  we  performed  multiple  two-sample  hypothesis  t-tests  on  the  frequency  of  alpha 
bursts  in  each  dataset.  We  tested  the  null  hypothesis  that  the  frequency  of  alpha  bursting  in  the 
training  and  validation  sets,  as  well  as  the  testing  and  validation  sets,  were  equivalent. 

2.3  Eye-tracking 

2.3.1  SensorMotoric  Instruments  Eye-tracking  System 

The  subject’s  eye  movements  were  recorded  by  the  SensoMotoric  Instruments  Eye-tracking 
System  with  sampling  rate  of  250  Hz  and  spatial  resolution  of  0.03°.  This  eye-tracking  system 
remotely  records  the  right-  and  left-eye-mapped  pupil  diameters  (RmapD  and  LmapD),  and  the 
gaze  measure  x-y  point-of-regard  (PORx  and  PORy)  in  real  time  using  a  camera  mounted  on  the 
simulator.  Pupil  diameter  and  point-of-regard  data  can  be  examined  using  plots  such  as  the  ones 
in  figures  9  and  10,  respectively.  In  figure  9,  the  horizontal  plane  represents  the  subject’s  field  of 
view.  A  frequency  plot  of  viewed  x-y  coordinates  reveals  how  the  gaze  was  distributed  across 
the  field  of  view  during  a  given  time  interval.  In  this  case,  the  gaze  is  concentrated  in  the  upper 
middle  region  of  the  simulation  screen.  In  figure  10,  both  low  and  high  frequency  components 
are  apparent  in  the  temporal  plot  (time  in  sec)  of  pupil  diameter  (mm).  Pupil  diameter  data  are 
often  examined  in  the  frequency  spectrum  rather  than  the  temporal  domain. 
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Figure  9.  Gaze  distribution  of  the  eye  on  a  2-D  screen.  The 

horizontal  plane  represents  the  subject’s  field  of  view. 
A  frequency  plot  of  viewed  x-y  coordinates  reveals 
how  the  gaze  was  distributed  across  the  field  of  view 
during  a  given  time  interval. 
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Figure  10.  Pupil  diameter  (mm)  over  time  (s). 


2.3.2  Pupil  Diameter 

Eye-tracking  was  measured  as  “pupillary  unrest”  similar  to  the  study  in  reference  26.  The  data 
were  interpolated  to  250  Hz  and  the  DC  offset  eliminated  by  subtracting  the  mean  from  each  data 
point.  Next,  the  data  were  separated  into  5-s  windows.  The  FFT  and  power  spectrum  were 
calculated,  and  the  spectral  power  between  0  and  0.8  Hz  was  measured  for  each  window 
independently.  Windows  with  gaps  greater  than  two  samples  (assuming  250  Hz)  were  labeled  as 
“bad  windows”  and  were  rejected  from  analysis.  Figure  11  illustrates  the  pupil  diameter  data 
processing. 
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Figure  11.  Flowchart  of  pupil  diameter  data  processing. 

2.3.3  Gaze  Distribution 

Gaze  distribution  was  measured  with  a  velocity-based  metric.  The  average  velocity  of  the  eye’s 
(x,y)  point-of-regard  was  calculated  for  every  5-s  window.  This  method  was  selected  to 
minimize  the  effects  of  differing  strategies  of  completing  the  simulation.  The  visual  field  actually 
required  to  successfully  complete  the  simulation  was  very  small,  as  the  speedometer  was  placed 
on  top  of  the  road  itself,  and  no  complex  maneuvers  requiring  foresight  were  required.  Fixations 
on  a  particular  area  of  interest  could  be  indicative  of  the  way  that  the  subject  chose  to  complete 
the  simulation  (simply  staring  at  the  speedometer  and  focusing  on  maintaining  the  exact  speed), 
or  of  actual  drowsiness  affecting  their  gaze  distribution.  This  factor  was  not  controlled  for,  so  the 
velocity-based  metric  does  not  favor  a  fixation  at  any  single  point  on  the  screen,  but  is  rather  a 
representation  of  the  eye’s  overall  movement.  Both  methods  would  be  confounded  by  this 
simulation  design,  but  the  velocity-based  method  averaged  across  windows  minimizes  its  effect. 
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For  quality-control  of  the  data,  all  (0,0),  (x,0),  and  (0,y)  points  were  removed,  as  they 
represented  missing  data  points  in  the  dataset  due  to  the  gaze  leaving  the  simulation  field  of  view 
or  the  camera  losing  sight  of  the  pupil.  Next,  intervals  with  a  sampling  rate  ranging  from 
240-260  Hz  were  used  exclusively  to  minimize  error  due  to  non-uniform  sampling.  This  data 
were  separated  into  5-s  windows  and  synced  to  windows  established  in  the  pupil  diameter  data. 
The  average  velocity  of  the  point  of  regard  was  returned  for  each  window,  and  then  windows 
with  less  than  20%  of  the  mean  window  sample  size  were  labeled  as  “bad  windows”  and  were 
rejected  from  analysis.  Figure  12  illustrates  the  gaze  distribution  data  processing. 


Gaze 

Distribution 


Figure  12.  Flowchart  of  gaze  distribution  data  processing. 


2.4  Behavioral  Indicators  of  Fatigue 

Behavioral  data,  recorded  at  90-100  Hz,  were  provided  by  the  Driving  Simulation  and  contained 
information  regarding  the  state  of  the  vehicle  in  the  simulation  at  a  given  point  in  time.  The 
driver  performance  metrics  recorded  by  the  simulator  included  X  and  Y  coordinates,  speed,  and 
acceleration.  In  addition,  the  simulation  generated  a  perturbation  that  distorted  the  heading  of 
the  vehicle  at  evenly  distributed  intervals,  which  the  driver  had  to  correct  for.  The  reaction  time 
of  the  driver  provided  an  additional  behavioral  metric  to  analyze. 

In  order  to  classify  this  behavioral  driving  data,  we  analyzed  them  using  two  different 
procedures:  the  first  methodology  used  regularly  spaced  windows  and  the  second  methodology 
separated  data  into  discrete,  event-specific  windows.  The  continuous  window  approach  involves 
analysis  of  driver  performance  metrics,  including  standard  deviation  of  lane  position  and 
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standard  deviation  of  acceleration,  in  continuous  windows  for  the  duration  of  the  experiment. 
The  discrete,  event- specific  approach  involves  analysis  of  driver  performance  metrics  in  the 
window  of  time  surrounding  a  specific  event,  in  our  case  the  reaction  time  to  a  perturbation 
generated  by  the  simulation.  For  a  graphic  depiction  of  the  two  approaches,  see  figure  13. 


Continuous  Window  Approach 


Alpha  Present 

Absent 

Absent 

Absent 

Present 

Absent 

(1) 

(0) 

(0) 

( 0) 

(1) 

(0) 

t  =  Os 

t  =  5s 

t  =  10s 

t  =  15s 

t  =  20s 

t  =  25s 

Event-Specific  Window  Approach 


Alpha  Burst  Present 
(1) 

Alpha  Burst  Absent 
(0) 

Perturbation  1  Onset 
(t  =  10.4  s) 

Start  of  Driver  Response 
(t=  Us) 

Figure  13.  Graphical  representation  of  two  approaches  to  windowing  driver  performance  data. 

2.5  Cross  Correlations 

To  analyze  inter-subject  variability  in  each  metric  we  used  for  analysis,  we  performed  multiple 
one-way  analysis  of  variance  (ANOVA)  tests  on  gaze  distribution,  pupil  dilation,  standard 
deviation  of  lane  position  (SDLP),  and  standard  deviation  of  acceleration  (SDA).  Each  ANOVA 
analysis  involved  only  one  of  the  aforementioned  metrics  and  compared  the  variability  of  each 
subject  with  respect  to  the  metric  being  analyzed.  To  prepare  analysis  of  the  correlations  between 
the  three  sensor  modalities,  we  first  had  to  sync  all  data  streams  to  a  universal  clock,  eliminating 
windows  that  were  not  in  sync  among  all  sensors  analyzed.  This  involved  performing  linear 
transformations  to  the  time  indices  of  the  data  that  accounted  for  both  a  constant  offset  time  and 
a  linear  drift  term.  In  addition,  windows  were  removed  from  all  data  streams  that  were  deemed 
“bad  windows”  for  either  the  pupil  diameter  or  gaze  distribution  data. 

In  order  to  find  the  cross-correlation  coefficients,  linear  correlation  analysis  was  performed 
between  each  of  the  eye-tracking  and  driver  performance  parameters:  gaze  distribution,  pupil 
diameter,  STDLAT,  and  SDA.  Due  to  a  lack  of  coinciding  alpha  bursts  with  “good  windows”  of 
eye-tracking  and  driver  performance  data,  no  cross-correlation  coefficients  were  calculated 
between  alpha  bursts  and  the  aforementioned  metrics. 
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3.  Results 


3.1  Detection  of  Alpha  Bursts 

3.3.1  Algorithm  Improvement 

Figure  14  illustrates  the  sensitivity  and  specificity  for  each  channel.  The  red  dot  indicates  the 
minimum  distance  to  the  vector  <1,  1>  (located  in  the  top  left  corner),  and  therefore  corresponds 
to  the  optimal  EEG  channel — Parietal-Occipital  7  (P07).  The  real-time  implementation  of  the 
algorithm  with  all  of  the  original  parameters  yielded  a  sensitivity  of  63%  and  a  specificity  of 
80%  for  the  P07  channel. 


Figure  14.  Graphical  representation  of  the  average  sensitivity  and  specificity  for  all  participants  for 

each  EEG  channel,  and  the  corresponding  location  of  each  EEG  channel.  The  P07  channel 
is  red  in  order  to  highlight  that  it  is  the  optimal  channel. 

Table  1  exhibits  the  remaining  algorithm  parameters  that  were  optimized  for  improved 
sensitivity  and  specificity  using  the  training  dataset.  The  overlap  between  windows  did  not 
change  through  optimization.  However,  the  window  length,  minimum  alpha  frequency,  and 
maximum  alpha  frequency  parameters  were  improved  through  this  optimization. 
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Table  1.  List  of  the  parameters  with  original  values  and  optimized  values  shown.  The  sensitivity  and  specificity 
(from  training  data)  for  each  parameter  before  and  after  optimization  are  also  included. 


Parameter 

Original 

Value 

Original 

Sensitivity 

Original 

Specificity 

Optimal 

Value 

Optimal 

Sensitivity 

Optimal 

Specificity 

Window 

Length 

1  s 

63% 

80% 

1.4  s 

66% 

76% 

Overlap 
(%  Window) 

75% 

64% 

76% 

75% 

64% 

76% 

Minimum 

Alpha 

Frequency 

7  Hz 

65% 

77% 

8.1  Hz 

65% 

81% 

Maximum 

Alpha 

Frequency 

13  Hz 

64% 

77% 

12.1  Hz 

67% 

78% 

3.3.2  Testing  and  Validation 

Table  2  displays  the  sensitivity,  specificity,  and  performance  coefficients  of  the  algorithm  in  its 
original  state,  when  it  is  implemented  in  real  time  with  the  original  parameters,  and  when  it  is 
implemented  in  real  time  with  the  optimized  parameters.  The  original  algorithm  yielded  a  high 
specificity,  but  low  sensitivity.  Implementation  of  the  algorithm  in  real  time  with  the  original 
parameters  caused  the  specificity  to  decrease  and  the  sensitivity  and  the  performance  coefficient 
to  increase.  Optimization  of  individual  parameters  yielded  an  improvement  in  the  sensitivity  and 
specificity  of  the  algorithm. 

Table  2.  Sensitivity,  specificity,  and  performance  coefficients  (from  testing  data)  for  all  three  iterations  of  the 
alpha  burst  detection  algorithm  are  shown. 


Original  Algorithm 

Original  Algorithm 
Implemented  in  Real-Time 

Optimized  Algorithm 
Implemented  in  Real-Time 

Sensitivity 

36% 

67% 

69% 

Specificity 

93% 

78% 

80% 

Performance 

coefficient 

71% 

73% 

75% 

As  described  above,  we  performed  two-sample  t-tests  on  the  frequency  of  alpha  bursting  in  each 
dataset.  Below  are  the  p-values  for  the  hypothesis  that  the  frequency  of  alpha  bursting  in  two 
datasets  is  equivalent  (i.e.,  the  below  p-values  are  indicative  of  the  probability  that  any  observed 
difference  in  the  frequency  in  alpha  bursting  was  due  to  random  chance.) 

•  Training  to  Validation:  0.53 

•  Testing  to  Validation:  0.75 

The  p-values  generated  by  the  t-tests  do  not  disprove  the  hypothesis  that  the  three  datasets  are 
statistically  identical. 
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3.4  Eye-tracking 

Figure  15  is  a  pictorial  representation  of  the  change  in  gaze  distribution  across  a  complete  test  of 
a  single  subject.  The  left  histogram  shows  the  beginning  third  of  the  data,  and  the  right  histogram 
shows  the  last  third  of  the  data.  The  horizontal  plane  is  a  representation  of  the  simulation  field  of 
view,  separated  into  super-pixels.  The  vertical  bars  represent  the  number  of  data  points  at  which 
the  eye  point-of-regard  was  in  that  super-pixel.  The  temporal  change  in  spatial  gaze  distribution 
is  obvious  for  each  subject,  but  differs  significantly  between  subjects. 


Figure  15.  Gaze  distribution  difference  over  simulation  duration.  The  horizontal  plane  is  a  representation  of  the 
simulation  field  of  view,  separated  into  super-pixels.  The  vertical  bars  represent  the  number  of  data 
points  at  which  the  eye  point-of-regard  was  in  that  super-pixel. 


3.5  Cross  Correlations 

The  analysis  performed  was  limited  to  gaze  distribution,  pupil  diameter,  standard  deviation  of 
lane  position,  and  standard  deviation  of  acceleration.  The  limited  analysis  was  due  to  limited 
overall  data  quantity.  Specifically,  this  was  the  result  of  both  malfunctions  in  the  eye-tracking 
system  and  an  inconsistent  clock  among  sensor  modalities.  These  problems  yielded  large  regions 
of  missing  and  incomparable  data  among  the  three  sensor  systems.  This  substantially  reduced 
the  amount  of  data  available  for  each  modality,  and  particularly  impacted  the  behavioral 
analysis.  In  addition,  of  the  remaining  data,  there  was  only  a  very  small  portion  that  was 
overlapping;  thus,  we  could  not  complete  cross  modality  comparisons  with  significant  statistical 
power,  one  of  the  initial  project  goals.  Recommendations  for  improving  data  quality  in  future 
experiments  are  included  in  a  separate  report  (32).  Only  the  continuous  window  approach  was 
used  for  the  results  detailed  below.  The  primary  method  of  analysis  used  for  these  measures  was 
linear  correlation  analysis. 

The  R  values  calculated  from  the  correlation  coefficients  between  eye-tracking  and  driving 
metrics  are  reported  in  table  3.  The  majority  of  the  coefficients  were  insignificant.  The 
significant  results,  90%  confidence  (yellow)  and  95%  confidence  (green),  indicated  very  low 
correlation  coefficients.  No  single  subject  or  combination  of  parameters  yielded  a  statistically 
significant  p-value.  Inter-subject  variation  seems  to  dominate  the  results. 


20 


Table  3.  R2  values  for  all  correlation  combinations.  The  green  cells  have  95%  confidence  (p<=0.05)  and  the 
yellow  cells  have  90%  confidence  (p<=0.1). 


Subject 

Gaze-Pupil 

Gaze-SDA 

Gaze-SDLP 

Pupil-SDA 

Pupil-SDLP 

SDA-SDLP 

11 

0.0000 

0.0047 

0.0000 

0.0007 

0.0073 

0.0069 

12 

0.0905 

0.0012 

0.0120 

0.0050 

0.0233 

0.0049 

13 

0.0190 

0.0136 

0.0000 

0.0153 

0.0002 

0.0001 

15 

0.0085 

0.0032 

0.0015 

0.0046 

0.0606 

0.0044 

16 

0.0001 

0.0005 

0.0596 

0.0026 

0.0017 

0.0004 

17 

0.0114 

0.0010 

0.0005 

0.0006 

0.0005 

0.0009 

18 

0.0015 

0.0004 

0.0039 

0.0001 

0.0010 

0.0130 

Note:  Gaze  =  saccade  velocity  measure  (eye  measure).  Pupil  =  Low  frequency  pupil  diameter  (eye  measure), 
SDA  =  standard  deviation  of  acceleration  (driving  measure),  and  SDLP  -  standard  deviation  of  lane 
position  (driving  measure). 


The  F  statistic  for  each  of  the  one-way  ANOVA  tests  was  large,  indicating  that  the  probability 
that  each  subject’s  data  (across  all  sensor  modalities)  comes  from  the  same  distribution  is  nearly 
zero.  This  indicates  that  non-parametric  statistics  are  necessary  in  order  to  combine  data  across 
subjects  in  analyzing  correlations. 

No  analysis  could  be  completed  regarding  the  correlation  of  alpha  bursts  with  eye  data  because 
none  of  the  alpha  bursts  coincided  with  “good  windows”  in  the  eye-tracking  data.  Therefore,  all 
of  the  data  containing  alpha  bursts  were  thrown  out  before  the  correlation  analysis. 


4.  Conclusions 


4.1  Detection  of  Alpha  Bursts 

By  adjusting  the  noise  approximation  methods  in  the  algorithm  in  reference  20  to  implement  a 
real-time  alpha  burst  detection  algorithm,  we  were  able  to  contribute  to  the  development  of  a 
fieldable,  EEG-based  drowsiness  detector.  Since  the  window  length  is  relatively  short  in  the  real¬ 
time  implementation  of  the  alpha  burst  detection  algorithm,  the  noise  present  within  each 
window  fluctuates  significantly.  One  might  expect  this  variation  in  noise  to  degrade  the  noise 
approximation.  However,  estimating  the  noise  on  a  window-by-window  basis  proves  to  be  more 
adaptable  to  differences  in  noise  between  subjects  and  with  time.  The  real-time  implementation 
yields  a  higher  performance  coefficient  than  the  original  algorithm,  in  addition  to  being  fieldable. 

Independently  optimizing  parameters  of  the  real-time  implementation  of  the  alpha  burst 
detection  algorithm  improves  the  sensitivity  and  specificity  of  the  overall  algorithm  as  well.  The 
algorithm  performed  better  on  the  P07  channel  than  on  any  other  channel.  This  is  consistent  with 
the  finding  that  the  primary  source  of  the  alpha  rhythm  is  in  the  posterior  and  occipital  regions  of 
the  brain  (13).  The  optimal  alpha  frequency  range  for  the  algorithm  was  8.1-12.1  Hz  (changed 
from  7-13  Hz).  These  changes  reflect  the  alpha  burst  frequencies  in  the  ground- truthed  data.  The 
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new  minimum  alpha  frequency  cutoff  is  consistent  with  most  views  of  the  alpha  frequency  range 
of  8-13  Hz  (13).  The  decrease  in  the  upper  limit  of  the  alpha  frequency  indicates  that  the 
majority  of  alpha  bursts  in  this  study  were  contained  within  the  lower  alpha  band  (8-10  Hz). 
Activity  in  the  lower  alpha  component  is  indicative  of  cognitive  processing  and  mental  effort 
(35). 

4.2  Cross  Correlations 

The  majority  of  the  correlations  between  driving  data  and  eye-tracking  measures  (reported  in 
table  3)  were  of  low  confidence.  Low  confidence,  where  alpha  is  defined  as  0.05  or  greater,  is 
indicative  of  correlations  that  cannot  be  statistically  differentiated  from  the  lack  of  a  correlation. 
While  some  metrics  for  some  subjects  appear  statistically  significant  in  our  results,  they  appear 
to  indicate  a  very  weak  correlation.  Additionally,  these  low-correlation-high-confidence  results 
appear  highly  subject- specific.  The  high  inter-subject  variability  in  eye-tracking  measures  and 
driving  performance,  as  indicated  by  ANOVA,  partially  helps  to  explain  this  result.  In  order  to 
perform  further  analysis  on  the  universal  correlations  between  these  reported  metrics,  non- 
parametric  statistics  must  be  used  to  account  for  inter-personal  variation.  In  additional,  a  larger 
quantity  of  clean  data  are  necessary  to  obtain  resolution  with  respect  to  the  many  low-confidence 
correlations  observed. 

With  respect  to  the  eye-tracking  data,  it  is  apparent  that  these  measures  cannot  be  used 
exclusively  for  drowsiness  detection.  Without  specific  measures  of  a  subject’s  position  in  space 
relative  to  the  eye-tracking  device,  head  motion  can  cause  significant  degradation  of  data  quality. 
Furthermore,  extended  periods  of  eye  closure,  which  are  associated  with  drowsiness,  can  result 
in  significant  data  loss.  For  these  reasons,  sensor  fusion  is  necessary  to  compensate  for  periods 
of  data  loss  and  is  the  most  effective  means  for  the  optimization  of  complex  physiological  state 
determination. 

4.3  Future  Work 

Given  the  limited  time-frame,  this  summer  project  is  a  small  piece  of  a  larger  effort  to  accurately 
predict  degradation  of  driving  performance  due  to  drowsiness.  In  order  to  further  improve  the 
alpha  burst  detection  algorithm,  the  integration  of  multiple  EEG  channels  should  be  investigated. 
A  central  decision  engine  could  treat  each  channel  as  an  individual  sensor,  which  would  yield 
more  accurate  results  than  would  a  single  channel.  In  this  distributed  sensor  scheme,  the 
transmitted  information  to  the  central  decision  engine  would  be  on  the  level  of  bits  per  time 
interval.  Therefore,  the  accuracy  of  the  algorithm  would  be  improved  without  sacrificing  power 
or  bandwidth.  This  setup  also  aims  to  improve  the  robustness  of  the  detector  by  giving  each 
alpha  burst  detector  a  relatively  small  weighting  in  the  decision  engine  algorithm. 

With  an  improved  dataset,  logistic  regression  could  be  used  to  analyze  the  correlation  of  alpha 
burst  presence  or  absence  with  the  continuous  metrics  representing  driving  performance  and  eye 
movement.  Additionally,  one  could  use  paired  sample  t-tests  to  test  the  hypothesis  that  alpha 
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burst  presence  is  associated  with  different  mean  driving  performance,  pupil  diameter,  or  gaze 
distribution.  Future  analysis  could  also  make  use  of  the  event-specific  window  approach  detailed 
above.  This  analysis  could  include  the  correlations  between  eye-tracking,  driver  performance 
measures,  and  alpha  burst  presence  with  respect  to  one’s  reaction  time  to  a  perturbation. 

The  continuation  of  this  project  should  include  the  addition  of  other  eye-tracking  measures, 
especially  PERCLOS.  Additionally,  EOG  recordings  provide  several  measures  like  blink 
amplitude/velocity  and  saccade  velocity,  which  have  the  potential  to  enhance  drowsiness 
assessment  and  driver  performance  prediction  capabilities.  Other  measures  indicative  of  the 
autonomic  nervous  system  like  skin  conductivity  and  measures  derived  from  heart  rate,  or  body 
orientation  data  are  also  viable  paths  forward,  but  as  these  parameters  were  not  investigated  as 
part  of  this  HRED  study,  their  inclusion  would  require  further  experimentation.  A  complete 
assessment  of  the  data  and  recommendations  for  future  experiments  can  be  found  in 
reference  32. 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


ANOVA 

analysis  of  variance 

ASD 

amplitude  spectral  density 

EEG 

electroencephalography 

EOG 

electrooculography 

FWHM 

full  width  at  half  maximum 

GAZEDIS 

gaze  distribution 

HRED 

Human  Research  and  Engineering  Directorate 

KSS 

Karolinska  Sleepiness  Scale 

NTSB 

National  Transportation  Safety  Board 

P07 

Parietal- Occipital  7 

PVT 

psychomotor  vigilance  test 

SDA 

standard  deviation  of  acceleration 

SDLP 

standard  deviation  of  lane  position 

STDLAT 

standard  deviation  of  lateral  lane  position 
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